Functional Approximations to Likelihoods/Posterior Densities: A Neural Network Approach to Efficient Sampling

The performance of Monte Carlo integration methods like importance-sampling or Markov-Chain Monte-Carlo procedures depends greatly on the choice of the importance- or candidate-density. Such a density must typically be "close" to the target density to yield numerically accurate results with efficient sampling. Neural networks are natural importance- or candidate-densities since they have a universal approximation property and are easy to sample from. That is, conditional upon the specified neural network, sampling can be done either directly or using a Gibbs sampling technique, possibly with auxiliary variables. We propose such a class of methods, a key step for which is the construction of a neural network that approximates the target density accurately. The methods are tested on a set of illustrative models that includes a mixture of normal distributions, a Bayesian instrumental-variable regression problem with weak instruments and near-identification, and a two-regime growth model for US recessions and expansions. These examples involve experiments with non-standard, non-elliptical posterior distributions. The results indicate the feasibility of the neural network approachMarkov chain Monte Carlo, importance sampling, neural networks, Bayesian inference

Are Education and Entrepreneurial Income Endogenous and Do Family Background Variables Make Sense as Instruments?: A Bayesian Analysis

Education is a well-known driver of (entrepreneurial) income. The measurement of its influence, however, suffers from endogeneity suspicion. For instance, ability and occupational choice are mentioned as driving both the level of (entrepreneurial) income and of education. Using instrumental variables can provide a way out. However, three questions remain: whether endogeneity is really present, whether it matters and whether the selected instruments make sense. Using Bayesian methods, we find that the relationship between education and entrepreneurial income is indeed endogenous and that the impact of endogeneity on the estimated relationship between education and income is sizeable. We do so using family background variables and show that relaxing the strict validity assumption of these instruments does not lead to strongly different results. This is an important finding because family background variables are generally strongly correlated with education and are available in most datasets. Our approach is applicable beyond the field of returns to education for income. It applies wherever endogeneity suspicion arises and the three questions become relevant.Education, income, entrepreneurship, self-employment, endogeneity, instrumental variables, Bayesian analysis, family background variables

Adaptive Mixture of Student-t Distributions as a Flexible Candidate Distribution for Efficient Simulation: The R Package AdMit

This paper presents the R package AdMit which provides flexible functions to approximate a certain target distribution and to efficiently generate a sample of random draws from it, given only a kernel of the target density function. The core algorithm consists of the function AdMit which fits an adaptive mixture of Student-t distributions to the density of interest. Then, importance sampling or the independence chain Metropolis-Hastings algorithm is used to obtain quantities of interest for the target density, using the fitted mixture as the importance or candidate density. The estimation procedure is fully automatic and thus avoids the time-consuming and difficult task of tuning a sampling algorithm. The relevance of the package is shown in two examples. The first aims at illustrating in detail the use of the functions provided by the package in a bivariate bimodal distribution. The second shows the relevance of the adaptive mixture procedure through the Bayesian estimation of a mixture of ARCH model fitted to foreign exchange log-returns data. The methodology is compared to standard cases of importance sampling and the Metropolis-Hastings algorithm using a naive candidate and with the Griddy-Gibbs approach.

Simulation based Bayesian econometric inference: principles and some recent computational advances

In this paper we discuss several aspects of simulation based Bayesian econometric inference. We start at an elementary level on basic concepts of Bayesian analysis; evaluating integrals by simulation methods is a crucial ingredient in Bayesian inference. Next, the most popular and well-known simulation techniques are discussed, the MetropolisHastings algorithm and Gibbs sampling (being the most popular Markov chain Monte Carlo methods) and importance sampling. After that, we discuss two recently developed sampling methods: adaptive radial based direction sampling [ARDS], which makes use of a transformation to radial coordinates, and neural network sampling, which makes use of a neural network approximation to the posterior distribution of interest. Both methods are especially useful in cases where the posterior distribution is not well-behaved, in the sense of having highly non-elliptical shapes. The simulation techniques are illustrated in several example models, such as a model for the real US GNP and models for binary data of a US recession indicator.

Bayesian estimation of the GARCH(1,1) model with Student-t innovations

This paper presents the R package bayesGARCH which provides functions for the Bayesian estimation of the parsimonious but effective GARCH(1,1) model with Student-t innovations. The estimation procedure is fully automatic and thus avoids the time-consuming and difficult task of tuning a sampling algorithm. The usage of the package is shown in an empirical application to exchange rate log-returns

Efficient Bayesian estimation and combination of GARCH-type models

This chapter proposes an up-to-date review of estimation strategies available for the Bayesian inference of GARCH-type models. The emphasis is put on a novel efficient procedure named AdMitIS. The methodology automatically constructs a mixture of Student-t distributions as an approximation to the posterior density of the model parameters. This density is then used in importance sampling for model estimation, model selection and model combination. The procedure is fully automatic which avoids difficult and time consuming tuning of MCMC strategies. The AdMitIS methodology is illustrated with an empirical application to S&P index log-returns where non-nested GARCH-type models are estimated and combined to predict the distribution of next-day ahead log-returns

Simulation based bayesian econometric inference: principles and some recent computational advances.

In this paper we discuss several aspects of simulation based Bayesian econometric inference. We start at an elementary level on basic concepts of Bayesian analysis; evaluating integrals by simulation methods is a crucial ingredient in Bayesian inference. Next, the most popular and well-known simulation techniques are discussed, the Metropolis-Hastings algorithm and Gibbs sampling (being the most popular Markov chain Monte Carlo methods) and importance sampling. After that, we discuss two recently developed sampling methods: adaptive radial based direction sampling [ARDS], which makes use of a transformation to radial coordinates, and neural network sampling, which makes use of a neural network approximation to the posterior distribution of interest. Both methods are especially useful in cases where the posterior distribution is not well-behaved, in the sense of having highly non-elliptical shapes. The simulation techniques are illustrated in several example models, such as a model for the real US GNP and models for binary data of a US recession indicator